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1. Introduction 


Improvement of life quality in the developed nations has systematically generated an 
increase in the life expectancy. A statistic studies curried out by the French national institute 
of statistic and economic studies (INSEE) shows a new distribution of age classes in France. 
In fact, almost one in three people will be over 60 years in 2050, against one in five in 2005, 
and France will have over 10 million of people over 75 years and over 4 million of people 
over 85 years. Nevertheless, the increasing number of elderly person implies more resources 
for aftercare, paramedical care and natural assistance in their habitats. The current 
healthcare infrastructure in those countries is widely considered to be inadequate to meet 
the needs of this increasingly older population. In this case a permanent assistance is 
necessary wherever they are, healthcare monitoring is a solution to deal with this problem 
and ensure the elderly to live safely and independently in their own home for as long as 
possible. 


In order to improve the quality of life of elderly, researchers are developing technologies to 
enhance a resident’s safety and monitor health conditions using sensors and other devices. 
Numerous projects are carried out in the world especially in Europe, Asia and North 
America on the home healthcare telemonitoring topic. They aim for example to define a 
generic architecture for such telemonitoring systems (Doermann et al., 1998), to conduct 
experiment of a remote monitoring system on a specific category of patients, like people 
with insufficient cardiac heart, asthma, diabets, patients with Alzheimer’s disease, or 
cognitive impairments (Noury et al., 2003)., or to build smart apartments (Elger et al., 1998), 
sensors and alarm systems adapted to the healthcare telemonitoring requirements (West et 
al., 2005). The project CompanionAble is an Integration Project founded by European 
commission (FP7). In this project we propose a multimodal platform for recognizing human 
activities of daily living (ADLs) in the home environment, by using a set of sensors in order 
to provide proactive healthcare telemonitoring for elderly people at home. This platform 
uses a fuzzy logic approach to fuse three main subsystems, which have been technically 
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validated from end to end, through their hardware and software. The first subsystem is 
Anason (Rougui et al., 2009) with its set of microphones that allow sound remote 
monitoring of the acoustical environment of the elderly. The second subsystem is RFpat 
(Medjahed et al., 2008), a wearable device fixed on the elderly person, which can measure 
physiological data (cardiac frequency, activity or agitation, posture and fall detection 
sensor). The last subsystem is a set of infrared sensors and domotic sensors like contact 
sensors, temperature sensors, smoke sensors and several other domotic sensors for 
environment conditions monitoring (Medjahed et al., 2008). This fuzzy logic approach 
allowed us to recognize several activities of daily living (ADLs) for ubiquitous healthcare. 
The decision of this multimodal data fusion platform is sent to a remote monitoring center 
to take action in the case of distress situation. 


2. CompanionAble project 


The CompanionAble project aim to provide the synergy of Robotics and Ambient 
Intelligence technologies and their semantic integration to provide for a care-giver's assistive 
environment. This will support the cognitive stimulation and therapy management of the 
care-recipient. This is mediated by a robotic companion (mobile facilitation) working 
collaboratively with a smart home environment (stationary facilitation). 


There are widely acknowledged imperatives for helping the elderly live at home (semi)- 
independently for as long as possible. Without cognitive stimulation support the elderly 
dementia and depression sufferers can deteriorate rapidly and the carers will face a more 
demanding task. Both groups are increasingly at the risk of social exclusion. 


The distinguishing advantages of the CompanionAble Framework Architecture arise from 
the objective of graceful, scalable and cost-effective integration. Thus CompanionAble 
addresses the issues of social inclusion and homecare of persons suffering from chronic 
cognitive disabilities prevalent among the increasing European older population. A 
participative and inclusive co-design and scenario validation approach will drive the RTD 
efforts in CompanionAble; involving care recipients and their close carers as well as the 
wider stakeholders. This is to ensure end-to-end systemic viability, flexibility, modularity 
and affordability as well as a focus on overall care support governance and integration with 
quality of experience issues such as dignity-privacy-security preserving responsibilities fully 
considered. 


CompanionAble will be evaluated at a number of testbeds representing a diverse European 
user-base as the proving ground for its socio-technical-ethical validation. The collaboration 
of leading gerontologists, specialist elderly care institutions, industrial and academic RTD 
partners, including a strong cognitive robotics and smart-house capability makes for an 
excellent confluence of expertise for this innovative project.3. State of the art 


Everyday life activities in the home split into two categories. Some activities show the 
motion of the human body and its structure. Examples are walking, running, standing up, 
setting down, laying and exercising. These activities may be mostly recognized by using 
sensors that are placed on the body (Lee et al., 2002). A second class of activities is 
recognized by identifying or looking for patterns in how people move things. In this work 
we focus on some activities identification belong to these both categories. 
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3.1 Data fusion 


In order to maximize a correct recognition of the various ADLs like sleeping, cleaning, 
bathing etc..., data fusion over the different sensors types is studied. The area of data fusion 
has generated great interest among researchers in several science disciplines and 
engineering domains. We have identified two major classes of fusion techniques: 


e Those that are based on probabilistic models such as Bayesian reasoning (Cowell et al., 
1999) and the geometric decision reasoning like Mahalanobis distance, but their 
performances are limited when the data are heterogeneous and insufficient for a correct 
statistical modeling of classes. 

e Those based on connectionist models such as neural networks MLP (Dreyfus et al., 
2002) and SVM (Burges et al., 1998) which are very powerful because they can model 
the strong nonlinearity of data but with complex architecture. 


Based on those facts the use of fuzzy logic in our platform is motivated by two main raisons 
from a global point of view: 


e Firstly the characteristic of data to merge which are measurements obtained from 
different sensors, thus they could be imprecise and imperfect. Plus the lack of training 
sets that reflect activities of daily living. 

e Secondly, Fuzzy logic can gather performance and intelligibility and it deals with 
imprecision and uncertainty. Its history proves that it is used in many cases which are 
necessary for pattern recognition applications. It has a background application history 
to clinical problems including use in automated diagnosis (Adlassnig et al., 1986), 
control systems (Mason et al., 1997), image processing (Lalande et al., 1997) and pattern 
recognition (Zahlmann et al., 1997). For medical experts it is easier to map their 
knowledge onto fuzzy relationships than to manipulate complex probabilistic tools. 


3.2 Fuzzy logic and patterns recognition systems 


Fuzzy logic is a fuzzy set theory, introduced by Lotfi A. Zadeh (Zadeh, 1978) in 1965; it is an 
extension of classical set theory. Historically, this was closely related to the concept of fuzzy 
measure, proposed just after by Sugeno (Sugeno, 1974). Similar attempts at proposing fuzzy 
concept were also made at the same time by Shafer (evidence theory (Shafer, 1974)) and 
Shackle (surprise theory (Shackle, 1961)). Since that time, fuzzy logic has been more studied, 
and several applications were developed, essentially in Japan. The use of fuzzy sets can be 
done mainly at two levels: 


e Attributes representation: It may happen that data are uncompleted or noisy, 
unreliable, or some attributes are difficult to measure accurately or difficult to quantify 
numerically. At that time, it is natural to use fuzzy sets to describe the value of these 
parameters. The attributes are linguistic variables, whose values are built with 
adjectives and adverbs of language: large, small, medium etc...and as an illustrating 
example, we found the recognition system proposed by Mandal et al. (Mandal et 
al.,1992). Some methods are based on a discretization of the attributes space defined as 
language. Thus a numerical scale of length will be replaced by a set of fuzzy labels, for 
example (very small, small, medium, large, extra large), and any measure of length, 
even numerical is converted on this scale. The underlying idea is to work with the 
maximal granularity, i.e. the minimal accuracy. 
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e Class representation: Groups do not create a clear partition of the data space, but a 
fuzzy partition where recovery is allowed will be better adapted. A significant number 
of fuzzy patterns recognition methods, are just an extension of traditional methods 
based on the idea of fuzzy partition for example the fuzzy c-means algorithm (Pedrycz, 
1990). Historically, the idea of fuzzy partition was first proposed by Ruspini in 1969 
(Ruspini, 1969). 


Rather than creating new methods of fusion and patterns recognition based on entirely 
different approaches, fuzzy logic fits naturally in the expression of the problem of 
classification, and tend to make a generalization of the classification methods that already 
exist. Taking into account the four steps of a recognition system proposed by Bezdek et Pal 
(Bezdek et al., 1992), fuzzy logic is very useful for these steps. 


e Data description: Fuzzy logic is used to describe syntactic data (Mizumoto et al., 1972), 
numerical and contextual data, conceptual or rules based data (Pao et al., 1989) which is 
the most significant contribution for the data description. 

e Analysis of discriminate parameters: In image processing, there are many techniques 
based on fuzzy logic for segmentation, detection, contrast enhancement (Keller et al 
1992) and extraction (Pal et al., 1986). 

e Clustering algorithms: The aim of these algorithms is to label a set of data into C 
groups, so that obtained groups contain the most possible similar individuals. Fuzzy c- 
mean algorithm and fuzzy ISODATA (Dunn, 1973) algorithm are the better known in 
this category. 

e Design of the discriminator: The discriminator is designed to produce a fuzzy partition 
or a clear one, describing the data. This partition corresponds to a set of classes. Indeed 
the fuzzy ISODATA algorithm is adapted for this step. 


3.3 Fuzzy logic steps 


We concentrate our efforts in emphasizing the fuzzy logic concept in order to integrate this 
fundamental approach within the telemonitoring platform. The main concept of fuzzy logic is 
that many problems in the real world are imprecise rather than exact (Buckley et al., 2002). It is 
believed that the effectiveness of the human brain is not only from precise cognition, but also 
from fuzzy concepts, fuzzy judgment, and fuzzy reasoning. An advantage of fuzzy 
classification techniques lies in the fact that they provide a soft decision, a value that describes 
the degree to which a pattern fits within a class, rather than only a hard decision, i.e., a pattern 
matches a class or not. Fuzzy logic is based on natural language which makes it quite 
attracting field in artificial intelligence. It allows the natural description of problem domains, 
in linguistic terms, rather than in terms of relationships between precise numerical values. 


A fuzzy set, as the foundation of fuzzy logic, is a set without a hard, clearly sharp defined 
boundary. A fuzzy set extends a standard set by allowing degrees of membership of an 
element to this set, measured by real numbers in the [0;1] interval. If X is the universe of 
discourse (the input space variable) and its elements are denoted by x , then a fuzzy set A 
on X is defined as a set of ordered pairs (x, /1,(x)) such that: 


A={x,u,(x)/x,0< ua(x)<1 } (1) 
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Where u(x) in equation (1), is the membership function (MF) of each x in A. In contrast 
to classical logic where the membership function _A(x) of an element x belonging to a set A 
could take only two values: w,(x)=1 if xe A or ua(x)=0 ifx gA, fuzzy logic introduces 
the concept of membership degree of an element x to a set A and w,(x)€[0;1], here we 
speak about the truth value. 


ED- 


Fig. 1. Fuzzy inference system steps. 


sindu 


=> Defuzzification 


sindjno 


A typical fuzzy logic inference system has four components: the fuzzification, the fuzzy rule 
base plus the inference engine, and the defuzzification. Figure 1 shows those main fuzzy 
inference system steps. 


3.3.1 Fuzzification 


First step in fuzzy logic is to convert the measured data into a set of fuzzy variables. It is 
done by giving value (these will be our variables) to each of a membership functions set. 
Membership functions take different shape. A Triangular membership function with 
straight lines can formally be defined as follows: 


0,x<a 
_ |(x-a)/(b-a),asx<b 
A(x,a,b,c) = EN foLb),b ANE (2) 
0,x 2c 


Trapezoidal function furnished in the equation (3). 


0,x<a 
(x-a)/(b-a),asx<b 
f(x,a,b,c,d)=51,b<x<c (3) 
(d-x)/(d-c)c<x<d 
0,x2d 


A Gaussian membership function with the parameters m and ø to control the center and 
width of the function is defined by: 
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—(x-m? 


G(x,m,o)=e 2° (4) 


The generalized Bell function depends on three parameters a, b, and c is given by: 


1 
1 +e-0) /a °° 


f(x,a,b,c)= (5) 


There are also other memberships functions like sigmoid shaped function, single function 
etc... The choice of the function shape is iteratively determined, according to the type of data 
and taking into account the experimental results. 


3.3.2 Fuzzy rules and inference system 


The fuzzy inference system uses fuzzy equivalents of logical AND, OR and NOT operations 
to build up fuzzy logic rules. An inference engine operates on rules that are structured in an 
IF-THEN format. The IF part of the rule is called the antecedent, while the THEN part of the 
rule is called the consequent. Rules are constructed from linguistic variables. These variables 
take on the fuzzy values or fuzzy terms that are represented as words and modeled as fuzzy 
subsets of an appropriate domain. There are several types of fuzzy rules, we only mention 
the two mains used in our system: 


e Mamdani rules (Jang et al., 1997) : which are on the form: If x1 is Ai and x2 is A2 and...and 
Xp is Ap Then y1 is C1 and y2 is C2 and...and yp is Cp Where A; and C; are fuzzy sets that 
define the partition space. The conclusion of a Mamdani rule is a fuzzy set. It uses the 
algebraic product and the maximum as T-norm and S-norm respectively, but there are 
many variations by using other operators. 

e Takagi/Sugeno rules (Jang et al., 1997): those rules are on the form : If xı is A1 and x2 is 
A2 and...and xp is Ap Then y = b otbi xı +b2 x2 +...+ bp Xp. In the Sugeno model the 
conclusion is numerical. The rules aggregation is in fact the weighted sum of rules 
outputs. 


3.3.3 Defuzzification 


The last step of a fuzzy logic system consists in turning the fuzzy variables generated by the 
fuzzy logic rules into real values again which can then be used to perform some action. 
There are different defuzzification methods; in our platform decision module we could use 
Centroid Of Area (COA), Bisector Of Area (BOA), Mean Of Maximum (MOM), Smallest Of 
Maximum (SOM) and Largest Of Maximum (LOM). Equations 6, 7, 8 and 9 illustrate them. 


n 
2 aaa 


7 (6) 
Dee Ha(X;) 


Zcoa = 


M 
ZBOA = Xm? Ha(%) = £ Ha(X;) (7) 
i=1 j=M+1 


www.intechopen.com 


A Fuzzy Logic Approach for Remote Healthcare 
Monitoring by Learning and Recognizing Human Activities of Daily Living 25 


N > 
Žan 


Zmom = "N (8) 


* 
i 


Zsom =min(x;) and Ziom = max(x;) (9) 


Where x; (i=1,2,....,N) reach the maximal values of w(x) 


4. The multimodal telemonitoring platform 


We define a smart environment as one with the ability to adapt the environment to the 
inhabitants and meet the goals of comfort and efficiency. In order to achieve these goals, our 
first aim is focused on providing such as environment. We consider our system as an 
intelligent agent, which perceives the state of the environment using sensors and acts 
consequently using device controllers. 


4.1 Sound environment analysis (Anason) 


In-home healthcare devices face a real problem of acceptance by end users and also 
caregivers. Sound sensors are easily accepted by care receivers and their family, they are 
considered are less intrusive then cameras, smart T-shirts, etc In order to preserve the care- 
receiver privacy while ensuring his protection and safety, we propose to equip his house 
with some microphones. In this context, the sound signal flow is continuously analyzed but 
not continuously recorded. Among different everyday life sounds, only some of them are 
considered alarming sounds: glass breaking, screams, etc. In order to have a reliable sound 
telemonitoring system, every sound event is detected (a sudden change in the 
environmental noise), extracted, and used as input for the classification stage. The sound 
analysis system has been divided in three modules as shown in Figure 2. 


The first module (M.1) is applied to each channel or microphone in order to detect sound 
events and to extract them from the signal flow. This module use an algorithm based on 
energy of discrete wavelet transform (DWT) coefficients was proposed and evaluated in 
(Rougui et al., 2009). This algorithm detects precisely the signal beginning and its end, using 
properties of wavelet transform. 


The second module (M.2) is a low-stage classification one. It processes the sound received 
from the first module (M.1) in order to separate the speech signals from the sound ones. The 
method used by this module is based on Gaussian Mixture Model (GMM) [14] (K-means 
followed by Expectation Maximization in 20 steps). There are other possibilities for signal 
classification: Hidden Markov Model (HMM), Bayesian method, etc. Even if similar results 
have been obtained with other methods, their high complexity and high time consumption 
prevent from real-time implementation. A preliminary step before signal classification is the 
extraction of acoustic parameters: LFCC (Linear Frequency Cepstral Coefficients) 24 filters. 
The choice of this type of parameters relies on their properties: bank of filters with constant 
bandwidth, which leads to equal resolution at high frequencies often encountered in life 
sounds. The best performances have been obtained with 24 Gaussians. 
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Fig. 2. Anason software architecture 


The sound classification module (M.3) classifies the detected sound between predefined 
sound classes. This module is based, also, on a GMM algorithm. The LFCC acoustical 
parameters have been used for the same reasons than for sound/speech module and with 
the same composition: 24 filters. A loglikelihood is computed for the unknown signal 
according to each predefined sound classes; the sound class with the biggest log likelihood 
is the output of this module. 


4.2 Vital signals wearable device (RFpat) 


The wearable device named RFpat (Hoppenot et al., 2009), designed by Telecom SudParis 
and integrated by ASICA, is devoted to the surveillance of the vital status of the care 
receiver, transmitting a fall index after validation by an embedded algorithm. Further 
functionalities of the wearable device include the eventual use of the emergency call button, 
the determination of the heart pulse rate (beat/ minute) and of a posture index, a movement 
frequency index and a technical status of the device. 
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Fig. 3. Internal structure of the wearable device (RFpat) 


In a case of emergency situation, for example if the care receiver has fallen down without 
standing up, with an eventual short delay, afterwards or has pushed the call button, the 
wearable device will transmit via ZigBee communication the corresponding alarm index to 
an in-home base station, which is connected to the multimodal platform. If no emergency 
event occurs, data are transmitted to this receiver every 30 seconds. In case of wireless link 
interruption, the data will be stored into an internal flash memory of the ZigBee transceiver 
and pushed through this ZigBee link when recovered. 


The device use two microcontrollers (Figure 3), the first is processing ”actimetric” sensors 
i.e. fall, movement and tilt sensor and driving analog switches used for the sampling process 
of the PPG signal pre-conditioner, the second being devoted to the processing of the pulse 
sensor. The ZigBee transceiver is also driven by the second microcontroller. All the circuits 
are supplied by a Lithium-Polymer battery element of 3.7 volts followed by 2 voltage 
regulators providing a voltage of 3 volts, one for the digital circuits and the ZigBee module, 
the second being used to supply the analog circuits. 


The vital signals terminal is planned as a mobile device worn by the person of care in the 
smart home environment as well as in the short range outside environment (garden etc.). 
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The mobile device is connected to the base station with a ZigBee network. The simple 
version of the network is working with two nodes. One node is defined as the coordinator, 
which is the base station on the central smart home control PC. The other node is defined as 
one end device, which is normally the wearable device. In poor RF conditions another node 
defined as a routing device that can extend the range between the base station and the 
wearable device. We have chosen the ZigBee IEEE 802.15.4 protocol because it is a secure 
and common protocol in the smart home environment. The most important advantages are 
the good power management and a good indoor wireless range with added routers if 
needed, which was preferred to a high bandwidth (WiFi for instance). We normally transmit 
3 bytes every 30 seconds. 


4.3 Home automation sensors 


The in-home healthcare monitoring systems have to solve an important issue of privacy. 
When developing our multi-modal platform, we chose the monitoring modules such that 
they have the less intrusive incidence on the monitored elderly person. We equipped our 
test apartment with wireless infrared sensors connected to a remote computer. The 
computer automatically receives and saves data obtained from the different sensors. Data 
corresponding to movements are collected twice per second, and stored with the event time 
in a specific file. 


The sensors are activated by the person’s passage underneath, and remained activated as 
long as there is movement under that sensor and for an additional time period of 1⁄2 seconds 
after the movement end. The results from the automatic processing of this data are 
displayed in the form of list with all movements noted together with the time and each 
movement's duration. This subsystem called Gardien is also able to display the data either 
in the form of graph (activity duration versus days) or as three-dimensional histograms 
(each sensor activation versus time). 


A set of wireless ambient sensors is added to this subsystem, they are designated for 
telemonitoring the environment of the patient and his surroundings. It includes state change 
sensors for active devices detection, contact sensors which are responsible for door and 
windows opening /closing detection, temperature sensors, fire sensors, flood sensors and 
light sensors. 


5. Fuzzy logic activities recognition approach 
5.1 Parameter and method elaboration 


The main advantages of using fuzzy logic system are the simplicity of the approach and the 
capacity of dealing with the complex data acquired from the different sensors. Fuzzy set 
theory offers a convenient way to do all possible combinations with these sensors. Fuzzy set 
theory is used in this system to monitor and to recognize the activities of people within the 
environment in order to timely provide support for safety, comfort, and convenience. 
Automatic health monitoring is predominantly composed of location and activity 
information. Abnormality also could be indicated by the lack of an activity or an abnormal 
activity detection which will cause or raise the home anxiety. Table 1 lists what we wish to 
automatically recognize. 
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ADLs of using devices ADLs of Human body motion 

getting up, toilet, bathing, sleeping, walking, standing up 

going out of home, enter home, setting down, laying 

washing dishes, doing laundry, exercising 


washing hands, watching TV, 
listening radio, cleaning 


talking on telephone, cooking 


Table 1. Fuzzy List ADLS to be recognized by the telemonitoring platform 


The first step for developing this approach is the Fuzzification of system outputs and inputs 
obtained from each sensor and subsystem. 


From Anason subsystem three inputs are built. The first one is the sound environment 
classification, all detected sound class and expressions are labeled on a numerical scale 
according to their source. Nine membership functions are set up in this numerical scale 
according to sound sources as it is in table 2. 


Membership Function Composition 
Human Sound snoring, yawn, sneezing, cough, cry, scream, 
laugh 
Speech key words and expressions 
Multimedia Sounds TV, radio, computer, music 
Door sounds door clapping, door knob, key ring 
Water sounds water flushing, water in washbasin, 
coffee filter 
Ring tone telephone ring, bell door, alarm, alarm clock 
Object sound chair, table, tear-turn paper, step foot 
Machine sounds coffee machine, dishwasher, electrical shaver, 


microwave, vacuum cleaner, washing machine, 
air conditioner 

Dishwasher glass vs glass, glass wood, plastic vs plastic, 
plastic vs wood, spoon vs table 


Table 2. Fuzzy sets defined for the ANASON classification input 


Two other inputs are associated to each SNR calculated on each microphone (two 
microphones are used in the current application), and these inputs are split into three fuzzy 
levels: low, medium and high. 


The wearable terminal RFpat produce five inputs; Heart rate for which three fuzzy levels are 
specified normal, low and high; Activity which has four fuzzy sets: immobile, rest, normal 
and agitation; Posture is represented by two membership functions standing up / sitting 
down and lying; Fall and call have also two fuzzy levels: Fall/Call and No Fall/ Call. 


The defined area of each membership function associated to heart rate or activity is 
adapted to each monitored elderly person. In our application we use only posture, and 
activity inputs. 
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For each infrared sensor C; a counter of motion detection with three fuzzy levels (low, 
medium, high) is associated, and a global one for all infrared sensors. 


The time input has five membership functions morning, noon, afternoon, evening and night 
which are also adapted to patient habits. 


For each main machine in the house a change state sensor S device,s name is associated. It 
has two membership functions turn on and turn off. One debit sensor for water is included 
in our application. Three membership functions characterize this sensor, low, medium and 
high. The output of our fuzzy logic ADL recognition contains some activities which are 
selected from the table I. They are Sleeping (S), Getting up (GU), Toileting (T), Bathing (B), 
Washing hands (WH), Washing dishes (WD), Doing laundry (DL), Cleaning (CL), Going out 
of home (GO), Enter home (EH), Walking (W), Standing up (SU), Setting down (SD),Laying 
(L), Resting (R), Watching TV (WT) and Talking on telephone (TT). These membership 
functions are ordered, firstly according to the area where they maybe occur and secondly 
according to the degree of similarity between them. 


The next step of our fuzzy logic approach is the fuzzy inference engine which is formulated 
by a set of fuzzy IF-THEN rules. This second stage uses domain expert knowledge 
regarding activities to produce a confidence in the occurrence of an activity. Rules allow the 
recognition of common performances of an activity, as well as the ability to model special 
cases. An example fuzzy rule for alarm detection is: 


If (Anason is Machine sound) and (Activity is motion) and (Coverai is high) and (Cg is high) and (Cs 
is high) and (Spacuum is turn on) Then (ADLs is Cleaning). 


A confidence factor is accorded to each rule and in order to aggregate these rules we have 
the choice between Mamdani or Sugeno approaches available under our fuzzy logic 
component. After rules aggregation the Defuzzification is performed by the centroid of area 
for the ADLs output. 


5.2 Software implementation 


Figure 4 provides a synoptic block-diagram scheme of the software architecture of the ADL 
recognition platform; it is implemented under LabwindowsCVI and C++ software. It is 
developed in a form of design component. We can distinguish three main components, the 
acquisition module, the synchronization module and the fuzzy inference component. 


It can run off-line by reading data from a data base or online by processing in real time data 
acquired via the acquisition module. To avoid the loss of data, a real time module with two 
multithreading tasks is integrated in the synchronization component. The platform is now 
synchronized on Gardien subsystem because of his smallest sampling rate (2 Hz) and 
periodicity. Indeed in some situations the RFpat system may be not used by the elderly 
person, namely if no recommendations relative to its cardiac watch or a particular risk of fall 
are given by the Doctor. 


The telemonitoring system with its Fuzzy tools allows the easy configuration of input 
intervals of fuzzification, the writing of fuzzy rules and the configuration of the 
defuzzification method. The general interface of the system allows to build up membership 
functions of inputs and outputs and displaying them. We could also write rules on text file 
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Inference 


Fig. 4. software implementation design 


by using a specific language, understandable by the telemonitoring system. This framework 
also allows for rules to be added, deleted, or modified to fit each particular resident based 
on knowledge about their typical daily activities, physical status, cognitive status, and age. 
The software implementation is validated with many experimental tests. The results and the 
rules which produced them are displayed on the main panel. 


6. DSS integration system 


The decision of this multimodal data fusion platform is sent to a real time decision 
integration system. This integration is performed by a multi-agent system (MAS) in which 
each agent coordinates separately with a decision support systems (DSS). The pertinence of 
each DSS is determined by the occurrence of false and undetected alarms. 


The agent delegates the decisional task to its corresponding DSS. The out coming decisions’ 
data are then formatted by the agent in an abstract decision report. This report format is 
recognized in the whole system and enables a central agent to make the final decision. A 
real-time negotiation of the decisions is able to improve the usage of appropriate resources 
within an acceptable response time. Thus, this multi-agent system architecture enables these 
DSS to have uniform view of the decision concept and to exchanges both knowledge and 
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intelligence, even if they implement several decisional techniques (Neural networks, fuzzy 
logic). In a remote healthcare monitoring system, we need such a solution in order to 
understand the behavior of the patient and the state of its domicile. Then, we can make the 
system evolve according to the analyzed behavior. 


6.1 Decision abstraction and priority assignation 


In intelligent remote healthcare monitoring, a decision support system uses the data flow of 
several modalities to generate decisions about the patient’s situation. To standardize the 
decision concept, we classify the generated decisions by the modalities used. The considered 
modalities in our system are: sound, speech, physiological data (e.g. activeness and pulse 
rate), actimetric data (localization, falls), video, sensor states and alarm calls. Generally, 
every decision is based on global pertinence calculated by combining the pertinence affected 
to each decision modality. For a d decision, the global pertinence is: 


Gp(d) = $ pid) ci (10) 


mi 


Where: m; is the modalities used for the decision d, p; is the pertinence of the decision d 
according to the modality m;, c; is the coefficient of the modality m; accorded by the DSS. 


When a DSS generates a decision, it sends the data concerning this decision to its 
encapsulating agent. The agent reorganizes these data in a decision report (type, pertinence, 
arrival date ...), which it then sends to the central agent. 


The collective decision is made in two phases: 


e Phase 1: the central agent starts the wait window of phase-1. The duration of the wait 
window depends on the trigger decision data (agent affinity, modalities used ...). In 
this paper, we do not detail the computing algorithm of the waiting duration. The 
decision messages received in phase-1 are called SEND decisions. A SEND decision is a 
spontaneous decision. It is not a response to a previous request. In the case of a trigger 
decision, we also define the pertinence threshold. The arriving decision reports during 
this first wait window are fused with the trigger decision. If the final decision’s 
pertinence surpasses the threshold, the decision is confirmed as an alert. If the wait 
window is terminated without attaining the pertinence threshold, the central agent 
starts the second phase of decision. 

e Phase 2: the central agent starts a new wait window. During this wait window, a real- 
time consensus is launched among the agents concerned by the trigger decision 
modalities. For this purpose, the central agent assigns to each concerned agent a 
consensus priority. This is computed as follows: 


pi(d) = >, Ay °C; (11) 


med 


Where mj; is the modalities used in the trigger decision d, cj is the corresponding coefficient 
for each modality, A; is the affinity of the agent i for the modality mj. 
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During this second wait window, the received message may be SEND decisions. As they do 
not concern the launched consensus, they are placed in the wait queue. The response 
messages are called CALL BACK decisions. At the end of the second wait window, the 
central agent computes the global pertinence of the received CALL BACK decisions. If the 
pertinence threshold is reached, the trigger decision is confirmed otherwise it is rejected and 
a learning procedure is sent to the responsible agent. In this article, we do not detail the 
inner learning procedure of such an agent. 


6.2 Real-time scheduling of the collective decision process 


One of the major problems in the field of multi-agent systems is the need for methods and 
tools that facilitate the development of systems of this kind. In fact, the acceptance of multi- 
agent system development methods in industry depends on the existence of the necessary 
tools to support the analysis, design and implementation of agent-based software. The 
emergence of useful real-time artificial intelligence systems makes the multi-agent system 
especially appropriate for development in a real-time environment (Julian and Botti, 2004). 
Furthermore, the response time of the DSS in a remote healthcare monitoring system is a 
central issue. Unfortunately the DSS studied in this context does not give a real-time 
response. For this reason we aim to control, as much as possible, the response time of their 
encapsulating Agents. The Gaia role model we presented in section 3 guaranties that the 
agent encapsulation of a DSS makes its response time transparent to the other agents. 


6.2.1 General operating principle 


This work has focused on a time-critical environment in which the acting systems can be 
monitored by intelligent agents which require real-time communication in order to better 
achieve the system’s goal, which is detecting, as fast as possible, the distress situation of the 
patient. The works of (Julian and Botti, 2004) define a real-time agent as an agent with 
temporal restrictions in some of its responsibilities or tasks. According to this same work, a 
real-time multi-agent system is one where at least one of its agents is a real-time agent. The 
central agent is the unique decision output of our system. We will apply these definitions by 
focusing on the real-time scheduling of the central agent tasks. Firstly the different tasks of 
this agent must be defined. Subsequently, diverse scenarios and the priority assignation 
rules may be defined. 


As explained previously, the central agent receives all the decision reports in the system. 
The first main issue is thus the scheduling of the treatment of these messages. For each 
decision received the central agent chooses the concerned agents and assigns a response 
deadline to each one, based on the degree of expertise of the concerned agent in the 
modalities used. We propose a scheduling model that enables the reaching of a consensus 
between the different concerned agents while respecting the defined response deadlines. 


6.2.2 Definition of the central agent tasks 


As described in figure 6, an agent has two main functions: conative and cognitive. In the 
case of the central agent, the cognitive function consists of communicating with the other 
agents. The conative function consists of making final decisions. 
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This classification leads us to this list of tasks assigned to the central agent: 


e Cognitive tasks: 

- Message reception: connection establishing and stream reading. 

- Message classification: according to the type of the request, this task classifies each 
message in the appropriate wait queue. 

- Entities representation: this task comes into play at each collective decision cycle. Its 
role is to keep the state of the other agents in the central agent memory, as well as that 
of the central agent itself. 

- Message send : connection establishing and stream writing 

e Conative tasks: 

- Request analysis and execution: this task executes the selected message requests. 
Generally it triggers another task of the central agent (representation, message send, 
decision) 

- Decision: this task maintains a fusion buffer in which the message execution task puts 
the decision message. When this task is activated, it adds all the un-executed messages 
in the highest priority wait queue to the fusion buffer. 

- Deadline assignation: this task assigns an absolute deadline to each message before 
classification 

- Message selection: this task selects the message to be executed from the message buffer. 

- Phase manager: this task is responsible for the transition between the collective decision 
phases. It comes into operation when a wait window is closed or when a collective 
decision is made. Its main role is changing the priority of central agent tasks. 


Each task is executed according to the automaton described in figure 6. 


Fig. 6. Execution states of the central agent tasks. 


6.2.3 Message classification 


The central agent message buffer consists of 3 different wait queues (WQ): the CALL BACK 
queue, for the CALL BACK decision messages, the SEND queue, for the SEND decision 
message and the Best Effort queue, for the other communication messages (decisions, 
service requests ...) 
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The BE queue is FIFO scheduled (First In First Out). There is no deadline or priority 
consideration in this queue. The CALL BACK and the SEND queue are EDF scheduled 
(George et al., 1996). EDF is the preemptive version of Earliest Deadline First non idling 
scheduling. EDF schedules the tasks according to their absolute deadlines: the task with the 
shortest absolute deadline has the highest priority. 


Each message deadline must be determined before being classified in a wait queue. For this 
reason the Deadline assignation task, the message classification task and the message 
reception task must be fused. In fact, when a message arrives, the message reception task is 
activated. It cannot then be preempted before assigning the message to its corresponding 
wait queue. 


6.2.4 Queue priority and message selection 


The message queues have dynamic priorities. This priority is assigned by a phase manager 
task. In phase-1, the SEND queue has the highest priority. In phase-2, the CALL BACK 
queue has the highest priority. While the message buffer is not empty, the message 
execution task’s state is Ready. When it passes to execution, it selects the shortest deadline 
message from the highest priority queue. During the wait window of phase-1, the received 
SEND must be executed first. Thus we assign the highest priority to the SEND queue. When 
this wait window is closed, the decision task gets the highest priority. The CALL BACK 
queue has the highest priority in phase-2. Thus a phase cannot be terminated until the 
corresponding wait queue is empty and all the received decisions fused. 


6.2.5 Global scheduling of the central agent 


The main scheduling algorithm of the central agent is FP/HPF. FP/HPF denotes the 
preemptive Fixed Priority Highest Priority First algorithm with an arbitrary priority 
assignment (Lehoczky, 1990). 
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Fig. 7. Real-time scheduling of the central agent. 
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In table 2, we present the priority evolution of each task during the different steps of the 2- 
phase collective decision (the higher the number, the higher the priority). The phase 
manager task always has the highest priority. In fact, it is responsible for changing the 
system phase and the priority assignation. 


6.2.6 Scheduling sample 


In figure 8, we present a scheduling sample in a system composed of a central agent (CA) 
and five other agents (A1, A2, A3, A4, A5). The red arrows represent the movement of the 
task to the ready state. Here we present the priority assigned to each task at the beginning of 
each phase. We suppose that the message wait queues are initially empty. 


Wait For Phase-1 Phase-2 
Task . - z : me. 
trigger wait Decision wait Decision 

reception 4 4 2 3 1 
Send 1 1 3 
decision 2 2 4 1 4 
execution 3 3 1 2 2 
phase manager 5 5 5 5 5 


Table 3. Priority variation of the central agent tasks 
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Fig. 8. Temporal diagram of a scheduling sample 


Our sample scenario goes through these stages: a trigger decision from A3 is received. The 
execution task treats the received trigger and then requests that the phase manager start a 
new collective decision process. The phase manager starts the first phase. It opens a new 
wait window and changes the priority of the CA tasks. During phase-1, two SEND 
decisions are received (from A1 and A4). The first wait window is terminated by the 
phase manager task. 


The highest priority is assigned to the decision task. The pertinence threshold is not reached. 
The phase manager task starts the second phase. The highest priority in this task is accorded 
to the send task in order to allow the CA to activate the consensus. 
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During the phase-1 decision process, the CA receives two SEND messages. The reception 
task is preempted because it has a lower priority. In phase-2, Al, A2, A4 and A5 are 
involved in the consensus (a choice based on the trigger decision modalities). A SEND and 3 
CALL BACK decisions are received (positive: A1 and A5, negative: A4). The final fusion 
reaches the pertinence threshold. Two learning procedures are sent to A4 and A2. We 
suppose that the message buffer is initially empty. 


The phase manager task is responsible for changing the priority of the central Agent tasks. 
We can observe on figure 8 the priority assigned to each task at the start of each new phase. 
The task manager is activated at the end of the wait windows to hand over to the decision 
task. At the end of its treatment, the decision task hands back to the phase manager task 
which starts a new phase by changing the priority of the other tasks. 


7. Experimentation and results 


The proposed method was experimentally achieved on a simulated data in order to 
demonstrate its effectiveness. This simulation gives very promising results for the ADLs 
recognition. Figure 9 shows results for a stream of a data. This fist study was devoted to the 
evaluation of the system by taking into account rules used in this fuzzy inference system. 
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Fig. 9. ADLs recognition experiment for a stream data. 
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The used strategy consisted in realizing several tests with different combination rules, and 
based on obtained results one rule is added to the selected set of rules in order to get the 
missed detection. With this strategy good results are reached for the ADL output (about 97% 
of good ADL detection). 


The experimentation described here is preliminary but demonstrates that ubiquitous, simple 
sensor devices can be used to recognize activities of daily living from real homes. The 
system can be easily retrofitted in existing home environments with no major modifications 
or damage. 


8. Conclusion 


In this chapter we have explore the cutting-edge research and technologies in monitoring 
daily activities using a set of sensors deployed in the house. The objective of the research is 
to provide a feasible solution for improving care for elderly people, while significantly 
reducing the healthcare cost. Focusing on the open problem of multiple persons monitoring, 
we have used an optimal set of sensors, design an algorithm for ADL recognition based on 
fuzzy logic, and implement a prototype. This approach provides robust and high accuracy 
recognition rate. Assisting elderly persons in place will benefit from the results of this 
research. The next objective of this research is to use these identification activities for 
building a model for measuring the home anxiety, that increases or decreases according to 
the detection activity and the state of each device in the home. 
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